SAIVT-QUT@TRECVid 2012: Interactive surveillance event detection

In this paper, we propose an approach which attempts to solve the problem of surveillance event detection, assuming that we know the definition of the events. To facilitate the discussion, we first define two concepts. The event of interest refers to the event that the user requests the system to detect; and the background activities are any other events in the video corpus. This is an unsolved problem due to many factors as listed below: 1) Occlusions and clustering: The surveillance scenes which are of significant interest at locations such as airports, railway stations, shopping centers are often crowded, where occlusions and clustering of people are frequently encountered. This significantly affects the feature extraction step, and for instance, trajectories generated by object tracking algorithms are usually not robust under such a situation. 2) The requirement for real time detection: The system should process the video fast enough in both of the feature extraction and the detection step to facilitate real time operation. 3) Massive size of the training data set: Suppose there is an event that lasts for 1 minute in a video with a frame rate of 25fps, the number of frames for this events is 60X25 = 1500. If we want to have a training data set with many positive instances of the event, the video is likely to be very large in size (i.e. hundreds of thousands of frames or more). How to handle such a large data set is a problem frequently encountered in this application. 4) Difficulty in separating the event of interest from background activities: The events of interest often co-exist with a set of background activities. Temporal groundtruth typically very ambiguous, as it does not distinguish the event of interest from a wide range of co-existing background activities. However, it is not practical to annotate the locations of the events in large amounts of video data. This problem becomes more serious in the detection of multi-agent interactions, since the location of these events can often not be constrained to within a bounding box. 5) Challenges in determining the temporal boundaries of the events: An event can occur at any arbitrary time with an arbitrary duration. The temporal segmentation of events is difficult and ambiguous, and also affected by other factors such as occlusions

Deep Reinforcement Learning based Patch Selection for Illuminant Estimation

Previous deep learning based approaches to illuminant estimation either resized the raw image to lower resolution or randomly cropped image patches for the deep learning model. However, such practices would inevitably lead to information loss or the selection of noisy patches that would affect estimation accuracy. In this paper, we regard patch selection in neural network based illuminant estimation as a controlling problem of selecting image patches that could help remove noisy patches and improve estimation accuracy. To achieve this, we construct a selection network (SeNet) to learn a patch selection policy. Based on data statistics and the learning progression state of the deep illuminant estimation network (DeNet), the SeNet decides which training patches should be input to the DeNet, which in turn gives feedback to the SeNet for it to update its selection policy. To achieve such interactive and intelligent learning, we utilize a reinforcement learning approach termed policy gradient to optimize the SeNet. We show that the proposed learning strategy can enhance the illuminant estimation accuracy, speed up the convergence and improve the stability of the training process of DeNet. We evaluate our method on two public datasets and demonstrate our method outperforms state-of-the-art approaches

Climate extremes and grassland potential productivity

The considerable interannual variability (IAV) (~5 PgC yr−1) observed in atmospheric CO2 is dominated by variability in terrestrial productivity. Among terrestrial ecosystems, grassland productivity IAV is greatest. Relationships between grassland productivity IAV and climate drivers are poorly explained by traditional multiple-regression approaches. We propose a novel method, the perfect-deficit approach, to identify climate drivers of grassland IAV from observational data. The maximum daily value of each ecological or meteorological variable for each day of the year, over the period of record, defines the \u27perfect\u27 annual curve. Deficits of these variables can be identified by comparing daily observational data for a given year against the perfect curve. Links between large deficits of ecosystem activity and extreme climate events are readily identified. We applied this approach to five grassland sites with 26 site-years of observational data. Large deficits of canopy photosynthetic capacity and evapotranspiration derived from eddy-covariance measurements, and leaf area index derived from satellite data occur together and are driven by a local-dryness index during the growing season. This new method shows great promise in using observational evidence to demonstrate how extreme climate events alter yearly dynamics of ecosystem potential productivity and exchanges with atmosphere, and shine a new light on climate–carbon feedback mechanisms

Meta-Network Analysis of Structural Correlation Networks Provides Insights Into Brain Network Development

Analysis of developmental brain networks is fundamentally important for basic developmental neuroscience. In this paper, we focus on the temporally-covarying connection patterns, called meta-networks, and develop a new mathematical model for meta-network decomposition. With the proposed model, we decompose the developmental structural correlation networks of cortical thickness into five meta-networks. Each meta-network exhibits a distinctive spatial connection pattern, and its covarying trajectory highlights the temporal contribution of the meta-network along development. Systematic analysis of the meta-networks and covarying trajectories provides insights into three important aspects of brain network development

Learning based image transformation using convolutional neural networks

We have developed a learning-based image transformation framework and successfully applied it to three common image transformation operations: downscaling, decolorization, and high dynamic range image tone mapping. We use a convolutional neural network (CNN) as a non-linear mapping function to transform an input image to a desired output. A separate CNN network trained for a very large image classification task is used as a feature extractor to construct the training loss function of the image transformation CNN. Unlike similar applications in the related literature such as image super-resolution, none of the problems addressed in this paper have a known ground truth or target. For each problem, we reason abouta suitable learning objective function and develop an effective solution. This is the first work that uses deep learning to solve and unify these three common image processing tasks. We present experimental results to demonstrate the effectiveness of the new technique and its state-of-the-art performances

Spatiotemporal Analysis of Developing Brain Networks

Recent advances in MRI have made it easier to collect data for studying human structural and functional connectivity networks. Computational methods can reveal complex spatiotemporal dynamics of the human developing brain. In this paper, we propose a Developmental Meta-network Decomposition (DMD) method to decompose a series of developmental networks into a set of Developmental Meta-networks (DMs), which reveal the underlying changes in connectivity over development. DMD circumvents the limitations of traditional static network decomposition methods by providing a novel exploratory approach to capture the spatiotemporal dynamics of developmental networks. We apply this method to structural correlation networks of cortical thickness across subjects at 3–20 years of age, and identify four DMs that smoothly evolve over three stages, i.e., 3–6, 7–12, and 13–20 years of age. We analyze and highlight the characteristic connections of each DM in relation to brain development